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.■ABSTRACT 

Federally sponsored educational research and 
developnent projects are not achieving their full potentials such of 
their material is of little or no practical utility, and much that is 
of value is not veil disseainated or is not readily available. The 
National Institute of Education night iaprove the quality of research 
and developaent by: (1) ezaaining the range of activities, products, 
and services that notr exist; (2) initiating research prograas which 
ezaaine the procedures for disseainatic&^jrnd adoption of these 
activities, products, and services; (3) aaking inproveaents for the 
short run in existing programs and institutions. (SK) 
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THE DtSSEMINATION OF EDUCATIONAL R&D PRODUCTS: RESEARCH 
AND POLICY ISSUES FOR THE FEDERAL GOVERNMENT 

Daniel Weller 

The Rand CorporatioHt Santa Monica, CaUfornld 

t. INTRODUCTION 

This paper discusses Issues related to federal governneni prog'^ams 
for the dlsseminat ion of educational R&D products, and sugg^^:;' 4 general 
policy and research guidelines In this area for the National i stitute 
of Education. 

Federal dissemination activities are spread throughout numerous 
agencies and offices that play some role In educational research and 
development, principally In Washington, but at the regional and local 
level as well, Including many federally sponsored experimental or 
"Impact** programs such as Experimental Schools or Title I. The formal 
Institutions of dissemination are largely centered In the National 
Institute of Education, in particular with the National Center for 
Educational Communication (NCEC), which manages the decentralized ERIC 
clearinghouse network, as well as other dissemination programs* The 
'^product" of most education research Is a document which presents the 
conclusions of a study or an evaluation. The product of actual develop- 
ment can be (a) a way of organizing and structuring some set of be- 
haviors that are designed to help make education more efficient or 
productive (e.g., PBBS, Individualized Instruction), (b) a physical 
product designed to accomplish the same objectives (e.g., a new science 
kit, a new building design), or (c) some combination of these (e.g., 
the Sullivan reading program). Most ^'physical** products are developed 
by commercial publishers or other segments of private Industry; many 
behavioral products are developed by these companies, but many are also 



Helpful comments on an earlier draft of this paper were made by 
Davfd Clark (University of Indiana), Arthur Cohen (ERIC Clearinghouse 
for Jun'or Colleges), and John Plncus (The Rand Corporation). 



developed by private and nonprofit research corporations under federal 
or local contract. The products of educational R&O are diverse, but 
most research does not lead to deve lopment*- I t leads to a written study 
or report that is rarely carried further. 

In addition to the products of research and development there 
exists something called educational Informat fon-^which Is neither 
research nor a developed product, but simply, as the name implies some 
kind of information about educat lon--atout research in progress, about 
practice, about debate and argument anKJng professionals on various 
topics, and so on. 

Federal dissemination efforts embrace all of these possibilities-- 
research products, development products (which Imply some preceding 
research), and information. 

This paper first summarizes key problems now faced by education 
practitioners who might wish to use RSD products or Information, and 
the factors that contribute to these problems. The paper then goes on 
to consider a general research and policy posture for NIE In this area, 
relevant research topics of interest, and the question of appropriate 
NfE policy for the short run. 
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M . THE PRESENT SITUATION* 

Practitioners who try to use educational R&D products and informa 
tion face three major problems. 

First, practitioners often find it difficult to Identify, locate 
and acquire potentially useful materials. District-level specialists 
in large school districts with well supported research libraries, and 
some practitioners who live In the vicinity of special Information 
centers, have somewhat less difficulty than others. However, sven for 
these somewhat special cases, and certainly for most practitioners, 
these difficulties are very real. 

Second, practitioners find It hard to get the help they need In 
order to overcome these difficulties. Many educators do not at first 
have a clear understanding of the relationship between their problems 



In reviewing current problems of access to educational RsD prod- 
ucts, we argue as If It can be taken for granted that most education 
practitioners are motivated to find and make active use of the best 
available Information. As we shall see, this assumption (and dissem- 
ination activities that raly on It) Is open to question. 
A* 

Education practitioners can be specialists at either the district 
or building level. Including administrators as well as specialists In 
testing and guidance, curriculum, or personnel training; or general Ists, 
meaning largely classroom teachers. However, the distinction between 
specialists and general Ists does not appear to have an Important bear- 
ing on dissemination renut rements , for the kinds of dissemination 
activities required to Meet the needs of the specialist and those re- 
quired to meet the needs of the general ist are sufficiently similar to 
make the distinction uninteresting from the perspective of federal policy 
[Greenwood and Weller, 1972]. 

The research community faces a number of related problems In 
attempting to use educational RSD products or Information. However, 
since the needs, resources and training of researchers (both basic and 
applied) are quite different from those of practitioners, this paper 
does not discuss these problems In detail. In practice It seems likely 
that federal policies designed to Improve dissemination for practitioner 
users will lead to improvements for the research community also. [Ibid.] 



and their requirements for R&D products or information. They may 
need assistance in defining problems clearly, or In understanding the 
character of the difficulties they have identified. Ever, with adequate 
problem definition, most practitioners are not well trained in the 
appropriate search techniques, and will requ' che assistance of a 
professional specialist in locating informat.v.» that might be useful. 

Finally, even with problems adequately identified, relevant search 
areas pinned down, and information in hand, most practitioners will 
need some assistance with the practical interpretation of the products 
they have found. They are usually being asiced to use either printed 
material or microfiche, in the case of a research report, or a printed 
manual of procedure in the case of developed materials. Printed 
materials, however, are necessarily somewhat abstract, and often fall 
to capture the specific character of the fmplementatlcn problems faced 
by practicing educators. Moreover, these materials are not interactive' 
if upon reading a report or research study a practitioner formulates 
additional questions for which answers seem required before he can talce 
action, he cannot query the printed page, and will usually look for 
additional assistance before acting. At present he finds this assls" 
tance difficult to obtain. 

Flnallyt the practitioner who has successfully Identified candidate 
R&D products related to his Interests finds that he often has little 
guidance as to their probable utility, reliability or validity, and 
must essentially make these judgments for himself on the basis of his 
<Mn Instincts* 



In California recently, the state legislature mandated the 
Introduction of program budgeting procedures In every school district, 
and hired a firm to develop detailed Implementation guidelines. The 
guidelines were duly circulated; local districts tnmedlately began 
searching for experts who could help them Interpret and Implement the 
new procedures. 



Hi. CONTRIBUTING FACTORS 

Some thought has b«en devoted to these and related problems in 
recent years, and It Is now possible to sutmnarize a number of reasonable 
explanations for this situation. What follt^ Is an attaint to provide 
such a list as the basis for further discussion of federal research and 
pol icy In this area. 

1. Many RSD products and imtch of what passes for educational 
Information are of little or no practical utility. Thus, while it is 
appropriate to consider possible changes In current dissemination 
activities, it Is Important as well that we understand the very real 
constraints on the system posed by the material It has to work with. 
Even the best dissemination system In the world wilt be of little use 
to anyone if it has little of consequence to disseminate. 

2. At the same time, research or development products of potential 
value to educators do exist, from evaluations of specific programs, to 
studies of the reliability and utility of achievement tests, detailed 
descriptions of new curricula, reports of practical solutions to 
adniinlstratlve problems, and dozen's of others. Thus, v^lle It Is a 
truism that there is great need for better research of direct utility 

to the practicing educatcr, there s.'c doubtless useful products in the 
system as well. However, there Is at present virtually no organizcid 
effort to distinguish these products from those which are less useful. 
For practical purposes, the dissemination system now treats all in- 
formation as having equal value. Hence, though the average practitioner 
may be poorly equipped for this task by training and temperament, and 
has little time to spare, he must invariably perform the screening and 
assessment function for himself. 

3. An enormous amount of educational research has been accomplished 
during the last several decades, leading to the publication of a great 
many books, reports, professional articles ar i occasional papers. How- 
ever, only a comparatively small effort Is m.ide today to review and 
synthesize research outcc^s across the various areas that might be of 



Interest and practical utility to professional educators. In part 
perhaps because of this low level of effort In research synthesis, there 
has also been little attempt to translate the results of research Into 
practical program advice for professional educators. 

4. We have not yet developed adequate procedures for the collection 
and validation of Information on th-j best of current practlce--al though 
It Is this kind of Information that educators rely upon most heavily— 
nor for the translation and extrapolation of successful local practices 
Into general policy guidance for the Implementation of educational pro- 
grams designed to meet the varied needs of different student popula- 
tions. 

5. The practitioner who tries to Identify and acquire a helpful 
research or development product faces multiple, partly redundant sources 
of Information with widely varying visibility and accessibility-federal, 
state, local and private— with virtually no coordination of disparate 
efforts. He can try district resources; If those do not yield results, 
he can look In his local public or university library. Dut he may also 
need to go directly, by mall or by phone, to an ERIC clearinghouse, a 
local Information Resource Center, a county-run curriculum lab, or to 
any one of dozens of other possible sources [see, e.g., Wanger, 1971 J. 

6. If the practitioner overcomes the problem of multiple and 
physically remote accass points, he must still cope with confusing 
arrangements for system entry from any given locale. He faces a pleth- 
ora of different Indices, card files, microfiche storage systems, tech- 
nolvjglcal aids and guides to Information. These various sources of 
entry to the Information system have little procedural or structural 
uniformity, for they were not created through system-wide agreements 



NCEC attempts to do some of this Job through Its Targeted Com- 
munications Program and support of various ERIC Clearinghouse Information 
Analysis products. While these efforts appear to be fairly popular with 
practitioners who are aware of them, their quality and scope are uneven, 
and they have not received a high level of federal funding support 
[System Development Corporation, 1972; Greenwood and Weller, 1972). 

Objections to the Idea that this, kind of work might prove valu- 
able are discussed briefly on pp. 18-19. 



on the ground rules for decisions pertalnfng to indexing strategies, 
selection of key descriptors, or subject matter partitioning. 

7. The practitioner who seeks the advice and Intercession of 
professional experts finds a system that Is often passive, that is not 
structured to respond in depth to direct l.iquiry from the user, and 
makes little attempt to anticipate information demand and consumption 
patterns. Marketing mechanisms— the Identification of client needs, 
and an active attempt to meet those needs--are rare. To some extent, 
this appears to reflect a system bias towa the research community as 
the client, and away from an operational mode of dealing directly with 
the practicing educator as the principal user. It could be argued that 
this Is In some respects a sensible way of deallm) with multiple and 
partly redundant resources, for researchers are bcttter able to wend 
their way through this complex system than are practitioner users. At 
the same time, this could be said to represent acquiescence in the 
creation of what i^ht be described as a "closed loop" for research 
Information, wherein the research community uses the system for 
assistance in the creation of new research results, which In turn go 
back Into the system and again to the research community. There are, 
to be sure, many breaks In the loop--appl led researchers do communicate 
directly with education practitioners and practitioners do have access 
of sorts. Nevertheless, the apparent system-wide bias against direct 
response to the practitioner user effectively "freezes out" many poten- 
tial practitioner clients from timely access to the Information they 
seek. 

Federal Government Policies and Assumptions 

We have discussed problems faced today fay practitioners who seek 
educational R&D products or information, and factors that contribute 



The new education extension agent program Is designed to remedy 
some of these problems, but It Is not yet clear what Its Impact will 
be, since there have not been many accompanying changes to related 
elements of the dissemination system [Sleber, Louis and Metzger, 1972]. 
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to these problems. We turn now to a discussion of federal government 
policies and assumptions that appear to have played an inH>ortant role 
In the creation and maintenance of present dissemination arrang^ents» 
We then go on to suggest sc^ resea'^ch and policy guidelines that might 
be considered in order to resolve these problems. 

Much of the present information dissemination system was Inherited 
by the federal government, either as it now exists or in some Incipient 
version, when the government first expanded its education activities 
significantly in the middle ^60s. The multiplicity of sources* for 
example, Is a natural consequence of the decentralization of education 
In the United States. When the federal government did Initiate dis-- 
semination activities of its own, it identified and moved to ameliorate 
an Important problem'^-the absence of a central collection and indexing 
agency for Information that did not find its way Into professional 
Journals, and was therefore lost forever to nK>st potential users. The 
limited sponsorship of seleced Information analysis products, such as 
bibliographies, research reviews and state-of-the-'art papers was also 
begun. These initiatives could not have been adequate to the taslc of 
overcoming the problems we have discussed above [Burchlnal, 19^1 • 

Federal policies to date have been characterized by a relatively 
low level of total effort, and by reluctance to pursue objectives 
that go much beyond the limited goal of providing a passive archival 
system. At the same time, practitioner demands on the system (requests 
for direct assistance) have led on the one hand to a variety of un-- 
coordinated efforts to respond ad hoc , and, more recently, to a more 
formal response through the creation of the education extension agent 
program. The picture that emerges on balance Is one of some confusion 
about the appropriate clients for federal programs and the appropriate 
objectives of federal policies. For example, while regional laboratories 
have been supported by the federal government In their efforts to 
develop Improved curricula for the public schools, federal efforts to 
disseminate Information about the results of these developmnnt programs 
have been relatively low key, and It is difficult for a practitioner 



today to acquire pertinent Information about this work by querying the 
federal dissemination system. 

While many decisions must of course await better Information about 
the nature of user needs and the best ways In which to meet them, the 
federal government has made relatively poor use of available knowledge 
about possibilities for lmprovement--from existing research studies, 
from the field operations of federal and other (state, county, private) 
components of the Information dissemination system, and from Informed 
judgments of professionals who work In the system. Regular system 
evaluation procedures have not been Implemented, and there has been 
no consistent program of research designed to lead to system Improve- 
ment. Federa! policymakers may have been either unaware of many of 
the probi ?ms discussed above, or unable to point to ways In which key 
problems could be resolved. 

The federal government appears to have been making some key as- 
sumptions that may be unwarranted. First, the government isppears to 
have assumed that there Is an abundance of useful educational Information 
and many good R&D products, in fact, this does not seem to be the case. 

Second, the government appears to have assumed that the passive, 
archival mode for the dissemination of R&D products can provide adequate 
access to the Information that Is needed, for both the research and 
practitioner communities. The assumption also seems unwarranted. 
Multiple and physically remote access points and nonuniform, poorly 
designed search tools have In fact made It quite difficult for both 
researchers and practitioners to have convenient access to the Informa- 
tion they seek. 

Finally, the government appears to have assumed that adequate 
access will In turn be sufficient to Insure that education decisionmakers 
at the local level will make use of existing products and Information In 



It now appears to be assumed that the provision of system-user 
Intermediaries such as education extension agents will further Insure 
adequate access for the practitioner community (see, however, footnote 
on p. 7t above) . 



order to Improve education. This assumption would appear to Ignore the 
realities of the actual Incentives of educators to make de- slons that 
could result In changes from current and accepted practices. 

If the argument presented so far Is valid. It Is not unreasonable 
to suppose that the Institutions and policies for dissemination that 
have emerged at the federal level, while useful In many ways, do not 
yet meet the real needs of educators today. However, they may provide 
the essential basis for moving to an Improved system In the years ahead. 



IV. NtE'S RESEARCH AND POLICY POSTURE 

Two things are clearly requl red--declslons and Information. 
Decisions are needed to elaborate a clear federal policy In this area; 
information Is needed to help validate, refute, or amend the hypotheses 
and assertions posited here and elsewhere about dissemination activities, 
and to provide fresh Insights Into ways in which improvements can be 
effected. The NIE might think In terms of four broad steps: 

1 . Develop a clearer understanding of the range of activities , 
products, and services that now exist . We now have, or can obtain In 
short order, most of the information we need In order to lay out the 
spectrum of existing practice In the dissemination of educational R&D 
products and information. The proper frame of reference is not just 
federal activities, but the broader perspective of all existing educa- 
tional information resources — federal, state, local and private. Before 
any subsequent decisions can be made, and before it will be possible to 
know where to direct research efforts most efficiently, we shoud try 

to have as clear an idea as possible of the range of services, resources, 
and Institutional arrangements now in existence. 

2 . Make an Initial assessment of ways in v/hich we would like to 
see these activities, products and services improved . In effect, this 
requires at least a tentative decision about what a national educational 
Information system might bes t look like. At one end of the spectrum, 
for example, something like the present arrangement might be viewed as 
most appropriate, in that the decentralized nature of educational 
practice may require decentralized, multiple, redundant sources of 
educational information. In this case, the cost of eliminating redun- 
dancy and confusion in the system would be viewed as exceeding the 
benefits that could be expected from such an effort. Alternatively, 

it might be argued that present arrangements are wasteful and inef- 
ficient, and that they hold little hope for providing practitioners with 
the information they will need In the years to come, particularly with 
anticipated growth in both the amount and quality of educational R&D 
products over the next decade. These judgments are certain to occasion 
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a good deal of hedt» and should indeed be the subject of widespread 
debate. Nevertheless, they are a key step, for judgments about desir- 
able directions for a national educational information system, while 
they will not be ifmYiutable, must surely be made, and made early, If 
federal programs that will inevitably be part of a larger set of 
activities throughout the country are to De rationally cast within the 
framework of explicit objectives for the nation as a whole. 

3* Make tentative decisions about the proper role of the federa l 
government In leading, sponsoring or Implementing programs designed to 
effect desired changes . This implies a requirement for analyses of 
the extent to which desired practices and services cannot or should 
not be provided elsewhere, a review of present and anticipated resources 
available to the federal government, and difficult judgments about what 
it would be appropriate for federal ambitions to embrace. On the one 
hand, for example. It could be argued that the federal government should 
not be Interested In any further expansion of Its presence to the local 
level, and that its activities should be restricted essentially to fund- 
ing state, county and local efforts to disseminate Information, together 
perhaps with some modest federal effort roughly equivalent to that which 
is being made today. Alternatively, It might be concluded that It 
would be wrong to count on local Initiatives to provide the necessary 
services in all cases, and that only ambitious federal programs will 
ensure uniform and adequate access to educational Information In every 
school district. 

^. Assess the utility of various ^nst I tutional arrangements at the 
federal level for accomplishing desired objectives . A decision about 
the institutional form for federal efforts follows logical ly--but must 
follow, not precede--the first three steps described above. Institu- 
tional structure should be dictated by antecedent decisions about 
federal objectives, and these decisions must In turn be preceded by a 
clear understanding of the kind of national information dissemination 
system that would best serve the requirements of both practitioners 
and researchers. 



ERIC 
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Each of the broad steps described above Implies a requirement for 
various kinds of Information about the best way to proceed. Much of 
this Information Is already available and can be pulled together In 
order to assist policymakers; much remains to be gathered. .We take 
up this topic below. 



V, MAJOR RESEARCH qUESTtONS 

We have suggested that a logical ^Irst step wK>uld be the develop* 
ment of a clearer understanding of the range of activities, products 
and services that now exist. Existing research studies and professional 
judgments could also be mined for insight and advice regarding federal 
programs In this area. In addition, a carefully thought out and co-- 
ordinated program of research, institutional design, and experimenta- 
tion could be Initiated over the next several years In order to better 
inform and continue to test tentative decisions that may be made In the 
Interim. Such a research program should address at least the following 
major questions : 

I , How do the incentives of education practitioners to seek , 
acquire and use educational RsD products and Information vary with 
variations in 

o the substance of the Information, 
o Information format and style, 
o product availability, 

o strategies for marketing information products^ 
0 practitioner roles in the education system, 
o pract it ioner object Ives (e.g., maintenance. Improvement, 
reform) , and 

o opportunities to put products and information to practical 
use? 

We can be reasonably certain that Incentives to seek and use 
educational Information and R&D products do not exist Independent of 
incentives and opportunities to make and Implement policy decisions 
about educational programs in the classroom, school, or district* These 
may be incentives and opportunities to maintain educational programs 
at an existing level of quality, to Implement new and imnroved programs, 
or to reform (or retain) inefficient practices. What matters Is that 
these incentives and opportunities often precede Incentives to acquire 
and use educational information, and can determine the extent of the 
information acquired, the kind of information sought, the speed with 
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which ft is desired, the format considered most appropriate, the sty le 
of its acquisition, and the uses to which it will be put. While we can 
say this much with reasonable certainty, we do not know the ways in which 
these variables interact. It is probable that incentives and opportuni-* 
ties are related to perceptions held by different actors of: 

o opportunities for professional advancement (status, esteem, 
income) ; 

o the risks that may accompany a decision to act or withhold 
action; 

o opportunities to pursue deeply held beliefs. 

Thus the need for and uses made of educational information and R&D 
products are related in some way to the capacity and wi 1 1 ingness to 
act In education, and the nature of this capacity—the Incentive 
structure, the opportunities, the origins of policy dectsions—ls some- 
thing we must know more about if information dissemination is to have 
a direct bearing on the process of educational change. 

2. What ^ Inst { tut ional arran ge ments ^ most efficient for 

meeting the range of product/service objectives selected by the federal 
government? We have argued that form should follow funct ion--but how 
can different functions best be performed? Research In this area w<«uld 
attempt to determine the most appropriate source of support for Informa* 
tion system components, the function and client focus for each component, 
the research and data collection responsibilities of the system, and 
the location of various components and subcomponents (e.g., centralized 
or decentralized locations for different functions). The nature of 
the services to be provided by the system will Influence the number, 
size and location of system access points for various clients, as well 
as the responsibilities ot system management and professional staff. 
Procedures for the management and coordination of system activities would 
have to be considered, as would system capacity for growth and renewal. 
System design would reflect not only decisions with respect to overall 
objectives, but technical judgments concerning the best way to maximize 
information processing and communication efficiencies, attract the most 
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skilled and dedicated staff, and develop the most effective relation- 
ships with other Institutions, both federal and nonfederal. We are 
some distance from being able to judge what shape such a system should 
take, what Its component parts should be, where they should be located, 
what Rianag«nent procedures should be considered, or what kinds of staff 
to look for. 

3. How should specialists of various kinds be used to assist 
practitioners to make optimum use of available R&D products and educa- 
t'onal in formation? Some practitioners today have access to Information 
specialists who mediate or negotiate their requests for Information In 
various topic areas, in addition, NCEC Is now planning to put a number 
of education extension agents In the field. In part to fulfill this 
function. We have not yet collected and analyzed the bulk of available 
information pertaining to existing services of this kind, and little 
atteti^t has been made to study the Impact on educational decisionmaking 
of the provision of varieties of services with personnel who have 
different kinds of training. We are therefore not yet clear about what 
kinds of people such specialists should be— what kinds of training they 
should have, what organizational affiliations or Institutional character 
they should adopt, and what kinds of services they should provide. We 
do not have a clear understanding of the criteria that should be applied 
to their selection, nor of the arrangements that should be made for their 
continued training. A university reference librarian Is an Information 
specialist; so Is an Information Resource Center specialist who has been 
to a special training program at a regional laboratory. What Is the 
difference between the kinds of services these two people can provide; 
how effective are they in their different roles; how much has their 
training cost; who can they best service? We should try to look at 
these and at other existing "personnel" models In order to get a better 
idea of the most desirable characteristics for this critical component 
of the dissemination system. 

^* Is It possible to translate research results Into practical 
policy guidance for educators? Educational R6D, like that In many 
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social policy fields, Is characterized by a division of functions and 
specialties which largely removes the research coiranunity from the 
responsibility for drawing operational implications from the results 
of its work. While some applied research does make a serious effort 
to take advantage of the results of basic research into human behavior 
and social organization, there remain as yet few Interdisciplinary 
attempts to design specific educational programs (together with detailed 
specifications for implementation) that are based directly upon a spec- 
trum of education and social science research results, and could be 
tested in the light of those results. It appears, for example, that 
research has not yet been able to Identify any particular combination 
of education resources that is consistently and unambiguously related 
to educational success [Averch, et al., 1972]. At the same time, 
because contradictory research results do exist, and because research 
tools are not adequately refined In many areas, nK>re information of a 
practical nature — testing the application of research hypotheses as 
they relate to operational programs— might help to eliminate some of 
this ambiguity. But It Is not clear that this goal, though It may be 
desirable, can be attained, it Is difficult to Imagine what the 
practical program Implications might be for many research findings. 
We need more work In this area in order to see whether or not guide- 
lines for the translation of research into practical programs can be 
established, and to refine ways In which program outcomes can be tested 
in terms of original research conclusions. One way in which to begin 
such work might be to devise Institutional efforts to bring basic and 
applied researchers together with the designers of educational programs 
In such a way that program designers are forced to test the theories 
and assumptions underlying their designs against the substantive 
knowledge and hypotheses of the research community. Since there is so 
much diversity of opinion and belief in the research contnunity with 
respect to fundamental issues of human behavior, and since this lack 
of consensus Is In part responsible for the vacuum Into which, program 
designers have moved, this would be an extremely difficult r-ask, but 
one that could be rewarding not only for program design, but for the 
research cwnnunlty as well. 
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5* Can vm establish reliable procedures for the widespread 
collection of Information on the best of current practical and the 
translation of this Information into usable models of exemplary 
practice? This kind of research Is ex:renwly difficult. At the least» 
such efforts might Include an attempt to provide descriptive syntheses 
of the systemic effects of different combinations of resource Inputs 
(including student characteristics), leaching processesi and organlza-- 
tional structures, infonr^tlon thus acquired might then be Inspected 
for our ability to extrapolate ^'general rules^' for program success under 
various ci rcumstances » and such rules vrauld In turn have to be trans- 
lated into practical implerrh^ntatlon advice for working educators. Even 
so, the settings for program replication wi 1 1 be largely unique^ imply- 
ing a requirement for the kind of expert assistance with program imple- 
mentation techniques that is not readily available today (and where 
research under 3 above might be of some assistance). 

Eventually, a broad Information collection effort might have to 
be initiated* based on decisions about relevant performance criteria, 
program characteristics of interest, and related matters. The machinery 
for such an information collection effort is not available and would 
have to be designed. Careful attention would have to be given to the 
nechanisms through which this information was to be aggregated, analyzed, 
synthesized, and translated into a product of some utility. 

When this research issue is discussed it Is con^nly objected 
that we cannot reach agreement on what '*good" practice looks like 
(multiplicity of objectives); that we could not identify it even if 
we agreed on what it was (crudeness of current measures); and that, in 
any case, current practice is mostly bad and not to be encouraged, so 
that this kind of wci k is not worth the effort. While these objections 
deserve to be taken seriously, they wou d anpear on balance to lend 
added support to the suggestion that much serious work of the kind 
described above r&nains to be done before It will be possible to Judge 
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the desirability of mounting expensive efforts to collect Information 
on current practice." 

6. How can technology be used more effectively to give both 
practitioners and researchers better and faster access to the existing 
knowledge base? One way In which an extensive set of dissemination 
services might be decentralized w(Hi}d be to create a decentralized 
technology, tying districts into the services of a regional Informa- 
tion net that could respond to requests from romote users. We do not 
yet know how--or if--thls could be accanpHshed, and this Issue deserves 
further exploration before final decisions are made about the location 
of federally managed or sponsored Information system access points. 
In addition, little has been done to experiment with ways In which the 
school building (meaning building administrators and teachers) could 
be tied directly to Information resources, it is widely understood 
that neither teachers nor principals have adequate time to use standard 
library resources, even when such resources are physically convenient 
and accessible. (This is quite apart from the time required at such a 
location to sort out the confusion of multiple points of entry to the 
Information system.) It has been argued that an attempt to tie schools 
directly to Information would not be worth the cost and effort. This 
may well be the case, but we knew of no experimental efforts to verify 
such assertions, indeed, what may be the case today may not be the 
case tomorrow, for If the NIE Is successful In many of Its objectives, 
and If better Information for practitioners can be developed by the 
research community, the products that may be available for teachers 
and principals may improve in quality substantially over the next 
decade. Should that be the case, might want to begin to experiment 
now with ways In which the school building could be given more direct 
access to those products In the years to c(»ne. 



It is hard to resist making the observation that many of the 
strongest and most eloquent attacks on current practice come from 
professionally successful, highly educated critics who are products 
of the public school system. 
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in addition, education researchars are not well served by present 
system technology, in particular, we cannot now collect and maice 
available to researchers around the country the growing mass of longi- 
tudinal educational and related social science data presently stored 
on computer tapes In a variety of federal, state, university and private 
data banks around the United States. Every new rssearch project that 
attempts to scan available data for new insights, or use data already 
collected as the context for comparison with fresh information, must 
start essentially from scratch to search the country seeicing access to 
various existing data files, tt Is not yet possible**- though it may be 
technologically feasible — for the researcher to have access to a wide 
variety of data without moving from a local computer terminal. Techno- 
logical options for providing these services also need to be explored. 

7. How can v» establish widely acceptable screening, rating end 
classification procedures for educational R&D produc_ts and informit lon? 
One serious problem now facing the potential user of educational R&D 
products !s that little attempt is made on his behalf to differentiate 
high-quality products frc»n those which are less useful. It seems 
obvious that rating procedures cannot simply assign one-dimensional 
classifications to R&D products; at the same time It Is not clear what 
these procedures should be, nor whether procedures can be agreed upon 
that will be acceptable to the broad mass of practitioner users as well 
as to researcherVr ..Fa^ly complex procedures may have to be devised— 
procedures that differentiate among various levels of Information 
utility for different purposes and for different users. A first-level 
screening might be fairly crude, seeicing only to establish whether the 
material is of sufficient professional quality overall to be accepted 
into the system. These might be comparatively easy judgments to make, 
as long as the bases for the judgments were made explicit and the 
professional credentials of the Judqes were widely accepted, in 
addition, rejected material would probably have to be placed In a 
separate Information pool for inspection by users who did not trust 
these judgments. After this initial screening, more refined and complex 
rating procedures might have to be instituted, so that material could 
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be rated, not along a scale of excellent to poor, but In a n^nner that 
Indicated the level of sophistication required of the potential user» 
the objectives of the research, the scope of the research conclusions, 
the professional Interests of users who viK>uld stand to benefit most 
from the product, uses to which the product has been put, user response 
to date, and so on. These categories remain to be devised and tested > 



The research topics described above can be approached In a variety 
of ways*, It Is not our purpose In this paper to present an extended 
discussion of research strategies or designs. One aspect of this 
research that does deserve special mention, however--and special con- 
sideration when designing a research strategy--ls the probable systemic 
and Interactive nature of various activities and outcomes of Interest. 
In particular, key Interactive effects may exist between Institutional 
arrangements for the dissemination of Information, the use of various 
kinds of specialists to assist practitioner and researcher clientele, 
procedures for the collection and validation of Information on current 
practice, and the uses of technology. Ambitious and cwnprehenslve 
research designs aimed at increasing our understanding of these Interac- 
tive effects may eventually be required in order to support Informed 
pol Icy decisions. 
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VU POLICIES FOK THE SHORT RUN 

We turn finally to the question of policies that may be most 
appropriate In the short run for the support or modification of ex-* 
isting programs and Institutions, for though NIE may initiate new 
research and experimentation in this area, the world is not likely 
to stand still. Three criteria suggest th^selves for weighing 
policies to be undertal^en in the immediate future: 

I. The policies should maximize the probability that the 
government will be In a good position to Implemertt policy recoim^nda* 
tions that may eventually flow from a concerdd program of research, 
design, and experimentation. 

2« in the event that these research, design, and experimenta- 
tion efforts prove disappointing, or in the event that research 
points to organizational, practical, political or economic consider*^ 
at ions that militate against the implementation of significant sys** 
tern change, short*run policies should at the same time serve the 
objective of improving the existing system. 

3» The policies should capitalize on existing resources* 

The application of these criteria to current options need not 
yield conflicting policy recommendations. On the contrary, by 
attempting to meet all three criteria the NIE may be able to place 
itself in a good position both for obtaining desired system changes 
In the long run, and for achieving needed short-run Improvements. 

The first criterion implies a requirement to (a) keep long-run 
policy options open and, (b) att^pt to build substantive, psycho- 
logical and political momentum towards the possibility of eventual 
comprehensive system change. The best way to achieve these objectives 
may also be the optimum way to meet the second criterion of achieving 
needed improvement in the short run: 

First, in order to keep long-run policy options open, the NIE 
would want to prevent existing programs from hardening into permanent 
Institutions, and existing momentums from growing out of control. 
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Thls suggests a policy » for example, of deliberately withholding 
long-term conmitments to existing components of the federal Informa- 
tion dissem-natlon system, and an emphasis on experimental variations 
with new components of that system In order to collect as much In- 
cidental Information as possible without making a cotranltment to any 
one variant. Clearly, this approach would also support (and in some 
respects be a prerequisite for) a policy of seeking short-run Im- 
provements to the existing system. 

Second, one way In which to build substantive, psychological, 
ani political momentum for eventual comprehensive system change 
would be to begin noM to make changes that will almost certainly be 
necessary In any case. Thus, whatever long-run policy recommenda- 
tions emerge from further research, It now seems reasonably likely 
that a number of steps, among others, will almost certainly be 
required, even assuming quite modest federal ambitions for the 
future: 

o More coordination, through federal leadership, of diffuse 

state, local, federal and private efforts, 
o Further consolidation of existing federal programs In 

order to eliminate redundancy and management inefficiencies, 
o Introduction of Improved screening mechanisms In order to 

cut down on the amount of low-quality material that Is collected 

and disseminated by the system, 
o More Intensive efforts to collect detailed Information on 

local practice, together with efforts to screen and classify 

that information for different varieties of users, 
o More emphasis on the review and synthesis of existing 

research. 

o Some effort to translate existing researc!i findings Into 
practical program Implementation advice for practitioners. 
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o Continuing attention to the improvanent of system manage** 

ment and organization, 
o Work on the Improv&nent of access and entry to the systefn — 

better search tools, more aggressive product marketing, and 

more visible entry points. 

These are also the kinds of changes one would probably want to 
make In order to improve the existing system. Here, policies that 
seem likely to be included In recommendations for the achievement of 
most long-run objectives probably support short-run improv«nent goals 
as wel 1 . 

Finally, one efficient way in which to approach the task of im- 
proving the existing system Is to capitalize on existing resources. 
The most Important existing resources are people and technology. 
There appear to be many bright, inventive people working In the 
field of educational Infonnation dissemination. These people have 
valuable experience, and they have been responsible for some of the 
most cogent and constructive criticism of the dissemination system. 
By involving the best people at an early stage In efforts at im- 
provOTMsnt— bringing them Into the debate and planning stages of the 
work, Insuring open lines of communication as work proceeds— the NIE 
would surely benefit from their knc»Arledge and experience. At the 
same time, it would be creating Important psychological and political 
momentum for potential long-run change among a constituency—the 
working professionals in the field— whose support for federal pol- 
icies will be of critical importance to the government. 

By exploring ways in which technology can be utilized to Im- 
prove the efficiency and responsiveness of the syston, the NiE might 
well be able to make Important system improvements In the short run 
while laying the groundwork ^or more ambitious and sophisticated 
technological changes In the future. For example. It Is almost 
certainly important to begin at an early stage to acquire more in- 
formation about certain kinds of man-machine interactions, for if 
long-run policy recommendations should include proposals for sub- 
stantial new federal investments In technology to serve both the 
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researcher and practitioner user, policymakers will need high- 
confidence estimates of the Impact and utility of various technolog- 
ical aids. To make these estimates they will require longitudinal 
data, and should not wait until after long-run objectives and policl 
have been thrashed out to begin collecting that Information. 

To recapitulate briefly, It Is the thesis of this paper that 
because of present serious difficulties with the effective dissem- 
ination of educational R&D products and Information to education 
practitioners, the NIE should: 

Undertake a carefully thought out program of research, 
design, and experimentation that will enable It to: 

(a) Step back and look at the national picture taken as a 
whole. 

(b) Decide how and why this picture differs from what 
policymakers would prefer to see, and what the federal 
government should and can do about It. 

(c) Acquire the necessary Information for federal policy 
decisions In the years to come. 

2. Implement parallel efforts to improve the existing system 
without making any long-term commitments to present or currently 
planned institutions or programs. 

3. Involve the best professionals now working in the system 
in designing and Implementing both long-run research and short-run 
improvement efforts. 

If properly planned and executed, these Initiatives should be 
mutually supportive, and the chances for obtaining beneficial long- 
run system change should be enchanced without sacrificing attention 
to inmediate problems. 
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